This invention relates to the field of software testing. In particular, the invention relates to testing of combined code changesets in a software product.
One of the tenets of successful agile software development is “real time” verification.
A coder can build and test their own individual changeset Δ1 independently of other coders' changesets Δ2, Δ3, Δ4, . . . , Δn. A changeset is a set of changes which should be treated as an distinct group (i.e., an atomic package, or multiple atomic packages) produced by a single developer source. A “real time” verification approach encourages better code quality upfront: as the coder gets feedback on his/her changeset alone, there is no room for doubt about the cause of regression failures.
In contrast, common development strategies where build and test follow a regular cadence (for example, once per day) commonly involve building and testing a “melting pot” of changes. These can interact in unusual ways leading to unexpected failures, and understandably coders do not want to duplicate effort investigating (potentially) someone else's problem. This culture unfortunately can lead to lower quality injected code and the need for full time roles to analyze builds and test results.
There are existing solutions to this problem. Known prior art provides the facility for coders to submit “private” builds containing only their changeset and, with a sufficiently powerful build engine definition, this can include appropriate regression test capability.
Unfortunately, many legacy software projects face major challenges. The product must often be built on multiple architectures and this relies on an extremely powerful and expensive regression capability built over many years. It is not possible to provide this “per Δ” verification capability as the resource required to support it is orders of magnitude more than that available.
Some improvements may be gained by optimizing the build and test cycles to be as fast as possible. Techniques include: build only changed code; only run tests that exercise changed code. However, these techniques only work well for small changes and may not offer enough of a speed gain to reach the “build per changeset” goal.
Therefore, there is a need in the art to address the aforementioned problems.